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ABSTRACT 

In the survey entitled "Measuring Output of 
Institutions of Higher Learning," survey part ici pants were requested 
to respond to 9 questions concerning the process of measuring the 
educational and research output of college and universities. The 
first 5 questions sought to ascertain which of the currently 
available measures of output are utilized by the individual 
institutions surveyed and for what purposes they are presently 
employed. The last 4 questions sought information concerning 
prevailing attitudes and opinions related to the desirability and 
feasibility of developing an objective, standardized measure of the 
output of universities. Comments concerning each question were 
solicited from participants and this analysis includes the more 
informative and representative responses received. (Author/HS) 
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Survey end Interpretation of Questionnaire 
Data Regarding Measuring the Output 
of Universities 

The currently available measures of educational output are defective in 
several respects. Theoretical inadequacies are evident in several of them, 
in that they fail to measure "output", focusing their analysis instead upon 
"inputs" or costs. Others, while actually concerned with measuring output, 
utilize non-universal bases for measurement, making valid comparison and 
analysis between institutions an almost impossible process. 

Developing a theoretically satisfactory measurement model poses a variety 
of difficulty problems. The practical implementation of such a process and 
its application to the widely diversified types of educational institutions 
existing in this country, presents different, although nontheless equally 
troublesome difficulties. The ambiguities inherent in these measurements 
interject obstacles to their practical application which in terms of 
aggregate and/or comparative analysis, seriously reduce their utility except 
for internal institutional evalutaion. 

Is there then a current need to develop a standardized means by which 
we can objectively measure educational output? If such a need does exist, 
is it possible to practically implement a measurement process which meets 
the aforementioned criteria of comprehensiveness and universal applicability? 
In order to obtain factual information on current attitudes concerning these 
questions, a survey was conducted in the latter part of 1971 and the 
beginning of 1972. 

The purpose of the survey was two-fold. First, it sought to ascertain 
which, if any, of the available measures of educational output are employed 
by American colleges and universities. In this regard, information was 
sought concerning l) the frequency of utilization of the various methods 
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discussed; 2) their application in terms of educational and research output; 
and 3) the purposes for which the data produced was employed, e.g. planning, 
budgeting, evaluation, etc. Second, the questionnaire endeavored to 
ascertain current attitudes concerning the desirability and feasibility of 
developing a standardized, objective measurement process of the output of 
institutions of higher learning. 

The survey, entitled, "Measuring Output of Institutions of Higher 
Laming" was sanctioned by the American Council of Education on November 4, 
1971. An abstract of the survey appeared in the January, 1972 issue of the 
Council's monthly publication, Report on Questionnaires . Any correspondence 
with the Council concerning the survey should refer to questionnaire QR 7167. 

The first problem encountered in conducting the survey was to decide 
to whom, in fact, it should be directed. Almost all of the currently 
conducted measurement of the output of universities is limited to internal 
institutional analysis. Within the various institutions this function is 
usually performed by, or under the auspices, of a high ranking member of the 
university's administration. Those institutions with sufficient resources 
often contain within their administrative framework a division or department 
of planning and institutional research, whose function includes developing 
and implementing whichever measures of output are utilized by that university. 
Similar types of analyses are performed in smaller, less affluent institutions 
either by the university’s comptroller or by a presidential aide. 

It is often these same personnel who decide which, if any, measures of 

r 

educational and research output should be utilized. Certainly if a 
standardized, objective measure is to be developed and implemented on a 
wide-scale basis, it must meet with the approval of these individuals. It 
is for these reasons that the decision was made to include in the survey 
those individuals presently responsible for directing and carrying out the 



measurement functions within our institutions of higher learning. The 
questionnaire was directed then to the president's comptrollers, and 
planners of one-hundred and twenty colleges and universities within the 
United States. 

In order to obtain a wide range of responses concerning current 
practices and attitudes, and in view of the divergent types of institutions 
of higher learning existing in this country, the following procedures were 
followed in selecting those universities to be surveyed. The nation "was 
divided into quadrants approximating the distribution of universities 
presently in operation. Each quadrant (northeastern, southern, central and 
western) included either twelve or thirteen states. Within each state at 
least one public, state financed institution was randomly selected. The 
remaining institutions (a total of thirty from each quadrant) were chosen 
in an effort to solicit responses from as wide a range of participants as 
possible. In this regard, the following factors of differentiation were 
utilized in the selection process: 

1) financing - public vs. private 

2) size of institution - based on enrollment 

3) at least two junior colleges were selected from each quadrant 

The following tables indicate the final breakdown, by quadrant, of 

those institutions to which questionnaires were sent, and their response to 
the solicitation for participation. Of the one hundred and twenty univer- 
sities and colleges solicited for participation, 72 (60%) participated, 

14 (12%) responded but did not participated, and 34 (28%) failed to respond. 
Those responding but not participating indicated the following reasons for 
their non-participation: 

1. Information requested not available - 6 

2. Policy restrictions preclude distribution of information 
requested - 2 



3. Budgetary restrictions preclude participation - 4 

4. No reason given for non-participation - 2 

Table A indicates the rather eoual distribution, by geographical 
location, of those institutions participating in the survey: northeast = 14 
south = 21; west = 20; central = 17. Table B shows the breakdown of 
participants by the size of the university or college, based on full-time 
enrollment. and by geographic location. 
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Survey - TabLe A 

Total Responses 

INSTITUTIONS SOLICITIED FOR PARTICIPATION AND THEIR RESPONSE Solicited = 120 



k 





O 

ERIC 



Survey - Table B 



PARTICIPANTS 

(By Quadrant and Size of Institution) 



Over 

Quadrant Less Tlian 1,000 1,000 - 5 ,<000 5,000 - 10 . 000 10,000 - 20,000 20,000 



Northeast 

Public 3 

Private 1 5 



1 



3 

1 



Weal 

Public 

Private 1 



2 3 

1 1 



6 4 

1 1 



Central 

Public 

Private 



1 3 

4 



4 4 

1 



South 

Public 

Private 



1 3 

4 5 



4 3 

1 



Totals 2 = 3% 18 = 25% 18 = 25% 18 = 25% 16=22% 



Public Institutions 

Participating = 44 = 61% 

Private Institutions 

Participating = 28 = 39% 

Total Participants = 72 = 100% 
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Analysis of Responses to the Questionnaire 



Survey participants were requested to respond to nine questions 
concerning the process of measuring the educational and research output of 
institutions of higher learning. The first five questions sought to 
ascertain which of the currently available measures of output are utilized 
by the individual institutions surveyed and for what purposes they are 
presently employed. The last four questions sought information concerning 
prevailing attitudes and opinions related to the desirability and feasibility 
of developing an objective, standardized measure of the output of universities. 
Comments concerning each question were solicited from participants and the 
following analysis includes the more informative and representative 
responses received. 
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Table No. 1 



CURRENTLY EMPLOYED MEASURES UTILIZED TO C A LCULATE E DUCATIONAL OUTPUT 

Survey Question No. 1 
And Response 



"Which, if any, of the following measures is presently employed by 
your institution as a means of calculating educational output? 

A. Number of student credit hours of instruction 
D. Number of degrees awarded 

C. Estimates of discounted incremental lifetime earnings of graduates 

D. Starting salaries of graduates 

E. Others" 



Total Response 

Public 

Private 



72 Institutions* 
44 Institutions 
28 Institutions 



NUMBER OF INSTITUTIONS EMPLOYING VARIOUS MEASURES 



Credit Hours 

Public 42 (95%) 

Private 25 (89%) 

Totals 67 (93%) 



Degrees Earnings of 
Awarded Graduates 



35 


(80%) 


1 (2%) 


22 


(79%) 


0 


57 


(79%) 


1 0 %) 



Starting 

Salaries Others** 

5 (11%) 7 (16%) 

0 3 fll%) 

5 (7%) 10 (14%) 



* 3 of the institutions surveyed (4%) do not attempt to measure educational 

output 
** included: 

1) Student-teacher contact hours 

2) Professional and leadership achievement of graduates 

3 ) Productivity of faculty 

4) Number of graduates continuing in graduate or professional schools, 

with quality inde :. 

5) Job placement of graduates 

6) Professional activities of faculty 

7) Follow-up studies of graduates 

8) Periodic alumni reviews. 
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Tabic No. 2 



CURRENTLY EMPLOYED MEASURES UTILIZED TO EVALUATE EDUCATIONAL OUTPUT 



hirvcy Question No. 2 
And Response 



"Which, if any, of the following measures is presently employed by your 
institution as a means of evaluating educational output? (Here, evaluating 
refers to using calculated measures as a means of decision making, e.g. to 
allocate classroom space to initiate or terminate specific programs, etc.) 

A. Number of student credit hours of instruction 

B. Number of degrees awarded 

C. Estimates of discounted lifetime earnings of graduates 

D. Starting salaries of graduates relative to those of other institutions 

E. Others" 



,v 3 of the institutions surveyed (4%) do not attempt to evaluate educational 
output. 

** included: 

1) Full-time equivalent students 

2) Full-time equivalent faculty 

3) Faculty-student ratios 

4) Priority of programs 

5) Job placement and success 

6) Number of majors and minors served by a department 

7) A.C.E. ratings 

8) National rankings of graduate programs 

9) Evaluations by consultants and planning committee 



Total Response 

Public 

Private 



72 Institutions* 
44 Institutions 
28 Institutions 



NUMBER OF INSTITUTIONS EMPLOYING VARIOUS MEASURES 



Degrees Earnings Starting 

Credit Hours Awarded of Graduates Salaries Others** 



Public 41 (93%) 

Private 25 




33 (75%) 
20 




0 

0 



0 



2 (5%) 



9 (20%) 
8 (29%) 
37 (24%) 



Totals 66 



53 (742- 



0 



0 
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Table No. 3 



CURRENTLY EMPLOYED MEASURES UTILIZED TO EVALUATE RESEARCH OUTPUT 

Survey Question No. 3 
And Response 



"By what moans does your institution attempt to evaluate research 
output of students, faculty and staff?' 

A. Number of publications, grants, copyrights, etc. 

B. Income from grants, copyrights, patents, etc. 

C. Others" 

Total Response = 72 Institutions* 

Public * 44 Institutions 

Private = 28 Institutions 



Public 

Private 

Totals 



NUMBER OF INSTITUTIONS EMPLOY I NG VARIOUS MEASURES 



Number of Publications, Income From Grants, 

Grants, Copyrights. Etc. Copyrights, Patents Others** 



36 (82%) 
19 (68%) 
55 (76%) 



23 (52%) 12 (27%) 

7 (25%) 8 (29%) 

30 (42%) 20 (28%) 



* 13 of the institutions surveyed (18%) do not attempt to evaluate voeriuvh output 

** Included: 

1) Quality of research work and publications 

2) Evaluations of publications by colleagues 

3) Benefits of research expressed in dollars (in terms of effect on 
industry, agriculture, etc.) 

4) Number of persons engaged in research 

5) Prestige of journals in which publications appear 
b) Contributions to university goals 

/) Independent evaluations by outside consultants 

8) Citations in the literature 

9) Number of theses and dissertations 

10 ) Prizes, awards, honors, and subsidized leaves for faculty 

11) A.C.E. ratings 
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Table No. 4 



CURRENT EFFORTS DIRECTED AT EVALUATING EDUCATIONAL OUTPUT IN MONETARY TERMS 

Survey Question No . 4 
And Response 



"Is any attempt made by your institution to evaluate educational 
output ( not costs) in monetary terms?" 



Total Response - 72 Institutions 
Public - 44 Institutions 
Private = 28 Institutions 



INSTITUTIONS EVALUATING EDUCATIONAL OUTPUT IN MONETARY TERMS 




No 



Public 4 (9&)* 

Private 0_ 

Totals 4 (6%) 



40 (m) 
28 (100%) 
6 8 (94%) 



* Three of four institutions responding in the affirmative to this 
question indicated that their efforts at evaluating educational output 
in monetary terms resulted from the implementation of planning-programming- 
budgeting systems . 



Table No. 5 



PURPOSES FOR WHICI£_MFASURES OF EDUCATIONAL AND/OR RESEARCH 
OUTPUT ARE CURRENTLY EMPLOYED 

I 




► 

l 



Survey Question No. 5 
And Response 

"For which of the following purposes is (are) measures of educational 
and/or research output currently employed? 

A. Budgeting processes - conventional 

B. Performance budgeting 

C. Planning - Programming - Budgeting Systems 

D. Cost - Benefit analysis 

E. Evaluations of Past Performance 

F. Evaluations of Efficiency 

G. Others" 



NUMBER OF INSTITUTIONS EMPLOYING OUTPUT MEAS1IRES FOR 
VARIOUS PURPOSES - PUBLIC VS. PRIVATE 

Total Response = 72 Institutions 
Public = -44 Institutions 
Private = 28 Institutions 



Evaluations 



Budgeting- Performance Cost - of Past Efficiency 

Conventional Budgeting P-P-B-S Benefit Performance Evaluations Others** 



Public 


38 (86%) 


12 


(27%) 


29 


(66%) 


12 


(27%) 


20 


(45%) 


20 (45%) 


3 


(7%) 


Private 


21 (75%) 


4 


(14%) _ 


14 


(50%) 


8 


(29%) 


11 


(39%) 


13 (39%) 


0 




Totale 


59 (82%) 


16 


(22%) 


43 


(60%) 


20 


(28%) 


31 


(43%) 


33 (43%) 


3 


w 



* 7 of the institutions surveyed (10%) reported no utilization of 
measures of educational and/or research output. 

** Included: 

1) Space utilization and requirements 

2) Evaluation of effectiveness 

3) Projections 



Table No. 5A 
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PURPOSES FOR WHICH MEASURES OF EDUCATIONAL AND/OR RESEARCH 
OUTPUT ARE CURRENTLY EMPLOYED 

Survey Question No. 5 
Response - Continued 



NUMBER OF INSTITUTIONS EMPLOYING OUTPUT MEASURES 
FOR VARIOUS PURPOSES - ANALYSIS ACCOKDING TO 
ENROLLMENT SIZE OF INSTITUTIONS 



Total Response = 
Enrollment: 

Less than 1,000 
1,000 - 5,000 

5.000 - 10,000 

10.000 - 20,000 
Over 20,000 



72 Institutions* 

= 2 Institutions 
= 18 Institutions 
= 18 Institutions 
= 18 Institutions 
= 16 Institutions 





Budgeting - 


Performance 


Enrollment 


Conventional 


Budgeting 


Less than 1,Q00 


1 (50%) 


0 


1,000 - 5,000 


13 (72%) 


3 (17%) 


5,000 - 10,000 


16 (89%) 


6 (33%) 


10,000 - 20,0u0 


16 (89%; 


3 (17%) 


Over 20,000 


13 (81%) 


4 (25%) 


Totals 


7 59“ (82%) 


16 (22%) 



Evaluations 







Cost - 


of Past 


Efficiency 


P- 


-P-B-S 


Benefit 


Performance 


Evaluations 


0 




0 


0 


1 (50%) 


11 


(61%) 


8 (44%) 


8 (44%) 


8 (44%) 


14 


(78%) 


3 (17%) 


8 (44%) 


9 (50%) 


6 


(33%) 


3 (17%) 


8 (44%) 


7 (39%) 


12 


(75%) 


6 (38%) 


7 (44%) 


8 (50%) 


43 


(60%) 


20(28%) 


31 (43%) 


33 (46%) 



* 7 of the institutions surveyed (10%) reported no utilization of measures 
of educational and/or research output. 

** Included: 

1) Space utilization and requirements 

2) Evaluation of effectiveness 

3) Projection j 





* 



Others** 

0 

0 

2 ( 11 %) 



0 




I 



I 

? 
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Table No. 6 

POTENTIAL USES OF MEASURES OF THE’ O UTPUT OF 
UNIVERSITIES 



Survey Question No. 6 
And Response 



"Which of the following do you consider to be the most important 
potential use of measures of the output of universities? 

A. Budgeting processes - Conventional 

B. Performance - Budgeting 

C. Planning - Programming - Budgeting - Systems 

D. Cost - Benefit Analysis 

E. Evaluations of Pest Performance 

F. Evaluations of Efficiency 

G. Others" 

Total Response = 72 Institutions 

Public = 44 Institutions* 

Private = 28 Institutions* 



NUMBER OF INSTITUTIONS INDICATING VARIOUS PURPOSES 
AS THE MOST IMPORTANT POTENTIAL USE OF MEASURES 
OF THE OUTPUT OF UNIVERSITIES 



Evaluations 



Budgeting - 
Conventional 


Performance 

Budgeting 


P_P_B-S 


Cost - 
Benefit 


of Past 
Performance 


Efficiency 

Evaluations 


Others** 


Public 1 (3%) 


7 (24%) 


22 (76%) 


8 (28%) 


1 


(3%) 


4 (14%) 


2 


(7%) 


Private 2 (10%) 


3 (14%) 


11 (52%) 


11 (52%) 


3 


(14%) 


9 (43%) 


0 




Totals 3 (6%) 


10 (20%) 


33 (66%) 


19 (38%) 


4 


®r 


13 (26%) 


2 


w 



* 15 public and 7 private institutions did not indicate a response to this 

question; percentages, then, are based on responses from 29 public, 21 private, 
and 50 total institutions. 

** Included: 

1) Projections and trend analysis 

2) Evaluation of effectiveness 









Table No. 7 



CURRENT NEED TO DEVELOP A STANDARDIZED MEASURE OF THE OUTPUT OF UNIVERSITIES 

Survey Question No. 7 
And Response 



"Do you believe a need currently exists to develop an objective, 
comprehensive, standardized means of measuring the output of institutions 
of higher learning?" 

Total Response = 72 Institutions 

Public = 44 Institutions 

Private = 28 Institutions 



INSTITUTIONS INDICATING A NEED FOR A STANDARDIZED 
MEASUREMENT PROCESS 





Yes 


No 


Not Respondinc 


Public 


36 (82%) 


8 (18%) 


0 


Private 


23 (82%) 


4 (14%) 


1 w 


Totals 


59 (82%) 


12 (17%) 


1 (1%) 



Table No. 7 A 



CURRENT NEED TO DEVELOP A STANDARDIZED MEASUR E 
OF THE OUTPUT OF UNIVERSITIES 



Survey Question No. 7 
And Response - Continued 



"Do you believe a need currently exists to develop an objective, 
comprehensive, standardized means of measuring the output of institutions 
of higher learning?" 



RESPONSE ANALYZED ACCORDING TO ENROLLMENT SIZE 



Total Response = 


OF INSTITUTIONS 
72 Institutions 


Less than 1,000 = 


2 


Institutions 


1,000 - 5,000 = 


18 


Institutions 


5,000 - 10,000 = 


18 


Institutions 


10,000 - 20,000 = 


18 


Institutions 


Over 20,000 = 


16 


Institutions 



Enrollment 


Yes 


No 


Not Responding 


Less Than 1,000 


2 (100%) 


0 


0 


1,000 - 5,000 


15 (83%) 


3 (17%) 


0 


5,000 - 10,000 


13 (72%) 


4 (22%) 


1 (6%) 


10,000 - 20,000 


15 (83%) 


3 (17%) 


0 


Over 20,000 


14 (88%) 


2 (12%) 


'6 




59 (82%) 


12 (17%) 


“ Tjn) 



ERiC 
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Comments to Question No. 7 



Comments received from survey participants regarding this question 
indicate the existence of three major attitudiual groups expounding divergent 
opinions concerning the need to develop a comprehensive, objective, 
standardized measure of the output of institutions of higher learning. 

The first group, composed primarily of large, public universities, 
and almost unanimous in indicating a need for such a standardized measurment 
process, might best be termed the 'accountability school". Proponents of 
this ! particular position are of the opinion that the increasing pressure 
being exerted by local, state, and federal bodies, and the public, will 
make the development of such a measure an absolute necessity. The 
attitudes of this group were succinctly summarized by the Director of 
Institutional Research of a large, public, state university: "In view 

of todays tight money situation for most institutions of higher education, 
and State legislatures' reluctance to increase appropriations, I think 
this is a necessity in order to justify resource requirements. We need 
to be able to make interinstitutional comparisons." 

The second, and considerably more diverse group, (in terms of 
enrollment size and financing) although favoring the development of some 
means by which to measure educational output, objects to a standardized 
process of measurement. Comments from this group cite the divergence of 
goals, community roles, and types of educational institutions as indicating 
the inappropriateness of a standardized measure of output. Proponents of 
this position advocate instead the development of several measures of 
output, suitable, in alternative combinations, to the needs of the various 
institutions. 



The third, and somewhat smaller group, opposes the immediate 
development of such a measure. The basis for their opposition, and the 
contentions on which it rests, were clearly stated by the Provost of a 
large, private, Southern university: 

"It seems to me that measures of the output of higher education, 
particularly a single, comprehensive measure, are premature. At present 
there are a number of instruments which could be used as measures of 
output - the GRE, CUES, the Institutional Functioning Inventory, head 
counts of students, ^ead counts of graduates, credit hour production. 

The last three of these are easy to obtain, and thus are most often used, 
but they are not sensitive to a whole range of issues - quality, style, 
appropriateness - with which academicians are usually concerned. The 
first three and others like them have not been in place long enough at 
enough institutions to see whether they will work effectively. 

Quantification and computerization are improving the operations of 
institutions of higher learning in various ways. But the burden of 
proof that these methods can improve decision-making in higher education 
still lies with those who claim they can. A premature convergence on 
some single method which does not satisfy those who must use it may cl o 
the general movement more harm than good. I believe the first step will 
be getting in place really sensitive and accurate information systems. 

Only then will we have the flexibility to try different measures of output 
to find the most valuable ones." 
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Table No. 8 



FEAS IBILITY OF DEVELOPING A STANDARDIZED MEASURE 
OF THE OUTPUT OF INSTITUTIONS OF HIGHER LEARNING 



Survey Question No. 8 
And Response 



"Do you believe a standardized, objective measure (in monetary terms) 
of the output of institutions of higher learning can be developed?" 





Total Response 

Public 

Private 


= 72 Institutions 
= 44 Institutions 
= 28 Institutions 






Yes 


No 


Not Responding 


Public 

Private 


23 (52%) 
9 (32%) 


21 (48%) 
17 (61%) 


0 

2 (7%) 


Totals 


32 (44%) 


38 (53%) 


2(3%) 



Comments to Question No. 8 



Several of the affirmative responses to this question were qualified 
by comments indicating the necessity of developing more than one 
standardized measure of output stated in monetary terms. Those advocating 
this position, while favoring the theoretical concept of a measurement in 
purely economic terms, are of the opinion that in order to initiate a 
truly comprehensive analysis process, supportive non-monetary measures 
would also be necessary. 

The comments received from those participants answering question 
No. 8 in the negative, centered their opposition on the supposed dangers 
surrounding any effort to measure output in non-educational terms. The 
Director of Institutional Research at a private Northeastern university 
wrote: 

"I cannot speak for the entire college on this matter, but it appears 
dangerous to measure output in non-educational terms. It is true that 
there are economic constraints within which a university must operate. 
However, it is undesirable to substitute monetary variables for genuine 
educational output measurements, nor is the situation saved by measuring 
convenient indices such as credit hours, which are low in relevance. 

100 credit hours in badly taught courses are not twice as good as 50 in 
those well taught.*’ 
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Table No. 9 



MEASUREMENT OF OUTPUT AS A THREAT TO TRADITIONAL 
GOALS OF HIGHER EDUCATION 



Survey Question No. 9 
And Response 



"If a standardized, objective measure (in monetary terms) of the 
output of universities could be developed, do you visualize it as a 
potential threat to the ''traditional" goals of higher education, i.e., 
to develop individuality and self expression?’ 





Total Response 


= 72 Institutions 






Public 


= 44 Institutions 






Private 


- 28 Institutions 






Yes 


No 1 


Not Responding 


Public 


18 (41%) 


25 (57%) 


1 (2%) 


Private 


8 (29%) 


14 (50%) 


6 (21%) 


Totals 


26 (36%) 


3 9 (54%) 


7 (10%) 



Comments to Question No. 9 



This question engendered the most numerous, diversified, and 
opinionated responses received from survey participants. Thooe 
respondents expressing the opinion that a standardized, objective measure 
(in monetary terms), if developed, might threaten the "traditional" goals 
of higher education, can be further subdivided into two distinct 
attitudinal groups. The first, in spite of visualizing such a measure 
as potentially threatening, or perhaps particularly for this reason, still 
favor the development of this type of analysis mechanism. Ihe attitude of 
this group was expounded by the President of a Northeastern, private, 
liberal arts college, who commented: "Yes - but there should be a threat 

to education - survival is at stake - education must become more business- 
like in measures of output and productivity." 

The views of those survey participants who visualize such a measure 
as threatening, and for that reason oppose the development of this type of 
mechanism, are represented by the following three responses, all received 
from high ranking administrators of relatively large institutions. 

"Even though such a measure may have been developed for other purposes, 
it will be viewed as a threat to the "traditional goals" of education by 
faculty, etc. Secondly, administrators have a history of using such 
measures for other than their intended purposes; they could be used as a 
method for coercing different institutions within a state, or different 
colleges or departments within an institution, to behave in some desired 
manner. Third, such a measure would indirectly pose a threat to the 
individual faculty member's security. This may have some effect on his 
creativeness, motivation, etc." 
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"It would perhaps destroy the pluralistic nature of higher education 
by exercising great pressure for conformity to a standard." 

"Any standardized, objective measure (in monetary terms) will, in the 
long run, develop incentive motives which may cause the potential learner 
to expend his time and erorgy in areas directed toward monetary gains rather 
than professional compsl.c-nce." 

The comments of the those respondents who support the development 
and implementation of a standardized, objective measure of educational 
output, and who do not view such as mechanism as threatening individual 
development and self expression, present equally elucidating and convincing 
arguments . 

"The 'traditional* goals of higher education must always be evaluated 
to ensure that they meet the needs of the present. No method is a threat 
if it does, in fact, help to accomplish the educational goals of an 
individual. Educational development in the present, depends so much 
on the availability of money that any reasonable method of objectively 
measuring higher learning output would be most helpful in the 
administration of funds for achieving -educational progress." 

And finally this comment from the Executive Secretary for Academic 
Affairs of a private, Church affiliated university: 

"...of the great myths of education, perhaps the greatest is that 
measures of output in the most quantified terms are inimical to the 
humanistic goals of the universities. It is the job of the academic 
planner to destroy these myths in constructive fashion. Measurement 
tempered by humanism is in itself an ideal." 
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Conclusions from Survey Data 



1. An analysis of the survey response to Question No. 1 indicates the 
widespread utilization of "credit hours" (93%) and "degrees awarded" 
(79%) as the primary means by which institutions of higher learning 
calculate their educational output. The high degree of acceptance 
afforded these two measures is at least partially attributable to 
the relative ease with which they can be both calculated and under- 
stood. This same simplicity of course, lies at the root of the major 
criticism of their utility - they provided us with only limited 
information. 

2. Although a total of 22% of the survey participants also calculate 
educational output by at least one means other than those mentioned 
above, no other single measure is employed by more than five of the 
universities surveyed. Only 7% of the institutions responding 
indicated a utilization of "starting salaries of graduates", while 
the use of a measure based on "discounted lifetime earnings of 
graduates" was reported by only one university. A list of other 
presently employed alternative means of calculating educational output, 
as noted by survey respondents, appears along with Survey-Table No. 1. 
Regarding these alternative measures, the survey responses indicate 

a somewhat higher frequency of utilization among public institutions 
and among those universities with enrollments in excess of 10,000 
students. 

3. The responses to Question No. 2 closely parallel those received to the 
preceeding question. "Credit hours" and "degres awarded" are the 
primary measures utilized by survey participants to evaluate 
educational output. 24% of the responding universities report the use 
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of at least one measure in addition to those mentioned above, in their 
evaluation processes. A listing of these alternative evaluative 
techniques appears, along with the overall response to this question, 
on Survey-Table No. 2. 

4. The responses to Question No. 3, regarding currently employed methods 
utilized to evaluate the research output of students, faculty and 
staff, reveal the insufficiency of commonly accepted, objective 
measures for this purpose. 93% of the respondents that attempt to 
evaluate rrsearch output measure the number of publications, grants, 
copyrights, etc. -producod by individuals, and 50% consider the income 
derived from research efforts. Almost all of the survey participants 
commenting on this problem indicated that, in addition to these 
evaluative techniques, some subjective analysis of research 
endeavors was also attempted. A list of these alternative and/or 
adjunctive measures is found, along with the general response to this 
question, on Survey-Table No. 3. 

5. Question No. 4 was designed to ascertain how many, if any, of the 
respondents currently attempt to evaluate educational output in monetary 
terms. Although four institutions replied affirmatively, none of these 
indicated by what means this task is accomplished. This fact, along 
with the lack of evidence (in the responses to Questions No. 1 and 

No. 2) of the utilization of any such measurement techniques, leads 
one to suspect that these institutions are in reality measuring costs or 
input, rather than output. In any case, as reference to Survey-Table No. 4. 
will clearly indicate, very little effort is currently expended in 
attempting to evaluate educational output in monetary terms. 
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The response to Survey-Question No. 6 indicates the wide variety 
of purposes for which measures of educational output are currently 
employed. Respondents reported that su "i measures are presently 
utilized in performing nine different budgeting and/or evaluation 
procedures. The most frequently performed functions included 
conventions] budgeting processes, planning-programming, budgeting 
sys terns, evaluations of past performance, and evaluations of efficiency. 

As reference to Survey Tables No. 5 >and 5A evinces, this breakdown 
of utilization of measures of output for various purposes holds 
for all groups surveyed, despite vast differences in financing and 
student enrollment. 

Question No. 6 sought information concerning the probable future use 
of measures of the output of institutions of higher learning. The 
response indicates that most university administrators surveyed are of 
the opinion that planning-programming-budgeting-systems and cost- 
benefit analysis are the most important potential uses of such measures. 
(See Survey-Table No, 6) Beth processes, of course, demand the utilization 
of accurate and meaningful measures of output, as well as information 
about inputs and costs. This trend toward the use of more sophisticated 
budgeting and evaluative techniques.. oould seem to indicate a present 
and growing need for equally sophisticated means of measuring the 
output of universiti ,s. 

The necessity of developing an objective, comprehensive, standardized 
means of measuring this output was further substantiated by the 
response received to Question No. 7. Despite several comments questioning 
the sagacity of utilizing a standardized measure for this purpose, the 



great majority of all groups of survey respondents nevertheless 
favor the developing of an objective, comprehensive measure of output. 

(See Survey-Table No. 7) 

9. Question No. 8 sought information concerning current attitudes 

regarding the feasibility of developing an objective, standardized 
measure (in monetary terms) of the output of institutions of higher 
learning. Although only 44% of the survey participants replied 
affirmatively, many comments indicated that if such a measure were 
utilized in combination with other, non-monetarv measures, it might 
provide a useful, and meaningful addition to educational economic 
analysis. The greater affirmative response among publicly supported 
universities is very lilcely the result of increasing economic pressure 
being exerted by both legislatures and the public upon intitutions 
of higher learning. For many universities, improved accountability and 
inter-institutional comparison are becoming an absolute necessity, and 
indications are that the urgency of the situation will increase with 
the passage of time. 

10. Question No. 9 sought to ascertain current attitudes regarding the 
feasibility of implementing a standardized, objective measure of 
universities' output. Certainly, if such a process is ever to gain wide 
acceptance, it must meet v?ith the approval of those whose work is to be 
evaluated. This question, then, inquired as to whether the development 
of such a measure would be a threat to the "traditional" goals of higher 
education, i.e., to develop individuality and self-expression. A 
majority (54%) were of the opinion that the attainment of these goals 
would not be adversely affected by the utilization of such a measurement 
process. Several respondents, in fact, commented that the development and 



implementation of a standardized, objective means of measuring 
edueatiooing output would compliment such goals. The conclusion 
that the implementation of this type of analytic tool is feasible 
is further supported by the feet that several of those who visualize 
sucii a me..'. h as a threat to "traditional" education would favor its 



impl-me ..: jo :icri tor eraetiy that reason. 
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